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(57) Abstract 



Systems and methods for the multispectral imaging of skin tissue enables automatic characterization of the condition of a tegion 
of interest of the skin (2), based on direct digital imaging of the region of interest or the digitization of color photographic slides of the 
region of interest, illuminated by appropriately filtered light Preferably, a digital image.at a low spectral band is automatically segmented 
and that segmented mask is used to segment the other images by a digital processor (12c). Parameters related to tfie texture, assymetry, 
blotchiness and border irregularities are also automatically estimated. The region of interest is automatically characterized by the digital 
processor, based on those parameters. The region of interest may include a skin lesion, in which case the piesent invention enables the 
characterization of the lesion as malignant or benign. 
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SYSTEMS AND METHODS FOR THE MULTISPECTRAL 
IMAGING AND CHARACTERIZATION OF SKIN TISSUE 

5 This application claims the benefit of U.S. Provisional Application Nos. 

60/039,218 and 60/039,407, both of which were filed on February 28, 1997, and 
are incorporated by reference, herein. 

FIELD OF THE INVENTION 
This invention relates to methods and systems for the computer 
10 controlled analysis of digital images of skin tissue at a plurality of wavelengths, 
which may include those outside of the red-green-blue bands. The methods and 
systems further include the automatic characterization of the condition of the skin 
tissue, based on automatically computed values of parameters which are functions 
of characteristics of the skin tissue, based on the digital Images. Skin lesions can 
15 be analyzed for determining whether the lesion is a melanoma, for example. 
Systems for digitally imaging and analyzing skin tissue are disclosed, as well. 

BACKGROUND OF THE INVENTION 
Melanoma is a usually fatal skin cancer, unless It Is detected and 
surgically removed in its earliest stages. Early detection of malignant melanoma is 
20 difficult because early melanomas, those having a Breslow thickness less than 1 
mm, share many diagnostic features vvith benign lesions, such as dysplastic nevi 
or atypical rtielanocytic nevi. 

To aid In the analysis of lesions, conventional photography, referred 
to as "clinical Imaging", has been used to Image the lesion for further study. The 
25 effectiveness of clinical Imaging can be compromised, however, by specular 
reflection by the skin. Polarizers have been used for polarized imaging, which 
minimizes specular reflection. 

Dermoscopy is another technique for examining skin, in which 
specular reflection is minimized. Dermoscopy also assists in clinically 
30 differentiating melanoma from its benign simulants by enabling the observation of 
features of pigmented melanocytic lesions that are not discernible by the naked 
eye. In demioscopy. the skin is made more transparent to light by providing an oil 
layer over the skin, in front of the optical system. A glass plate is placed over the 
oil layer. The oil has an index of refraction between the index of refraction of the 
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horny layer of the skin and the glass plate. Standard magnifying optics may be 
used to enlarge the structures rendered visible on and under the surface of the 
skin by the oil layer. The region of interest can then be examined visually. Slides 
of the region of interest can be made, as well, for future study. 
5 Despite their similarities, most malignant melanomas differ in certain 

of their characteristics from other melanocytic lesions. A major advance in 
characterizing skin lesions based on certain of the observable differences between 
malignant and other lesions is the "ABCD" rule, where A=asymmetry, B=border 
inregularity, C=color variability, and D=diameter greater than 6 mm. A 

10 conresponding ABCD rule, where "D" refers to demroscopic structures, such as 
brown globules, black dots or pigment networks within the lesion, is applied to 
dermoscopic images. Because the clinical and dermoscopic applications of these 
rules are subjective, they are not very reliable. 

When skin is illuminated by light, the light can be re-emitted by 

15 reflection, scattering or fluorescence. It is known In the art that the re-emission of 
light absorbed at different \ravelengths by a region of interest of skin can provide 
different information. For example, as the wavelength of the light increases^ 
depth of penetration into the skin or other tissue also increases. Chromophores at 
different depths In the tissue therefore absorb and re-emit light at various 

20 wavelengths. Melanin and hemoglobin are examples of such chromophores. 
In addition, since the unaided eye cannot perceive light outside of the visible 
region or low-contrast structure in visible-light images, information which may be 
useful in diagnosing a lesion may not be directly observable. Digital acquisition 
and processing of dermoscopic images may. therefore, improve diagnostic 

25 reliability by employing more of the information residing in such images that is not 
directly observable. There have therefore been attempts to use objective, 
computer-based, image analysis algorithms that can discern meaningful 
differences between benign and malignant melanocytic lesions with sufficient 
accuracy. 

30 Computer processing of Images requires that the image be in digital 

form. A digital Image Is an array of digital signals whose values are a function of 
certain characteristics of the subject of the image. When imaging skin lesions, the 
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digital images comprise digital signals whose values are a function of the re- 
emission characteristics of the skin and lesion, at the spectral bands of the light 
illuminating the skin. The array of numbers is obtained by spatial sampling and 
quantizing the intensity of images obtained with film or directly by electronic 
5 cameras. Practical limitations on the number of picture elements or pixels per unit 
area of image determine the achievable spatial resolution of the digital image. 
The digital image typically needs to be segmented to separate the digital signals 
which are a function of the skin lesion from the digital signals which are a function 
of the surrounding skin. 

10 Computer aided analysis has also been used to classify skin lesions 

using quantitative values Indicative of particular characteristics of lesions, referred 
to as parameters. Based on histopathological diagnosis of lesions, algorithms 
have been developed which use linear or non-linear classifiers to combine 
parameters provided by the operator of an imaging device or a physician or 

15 computed by a processor, to yield a value which can be used to classify the lesion. 
Because some of the steps in the computer-aided analysis of which we are aware 
to date depend on subjective Judgments of an individual, such analysis may 
provide highly variable results. 

The images heretofore available have been obtained with 

20 commercially available red-green-blue color imaging apparatus. Color 

photographic transparencies of skin lesions have been digitized and skin lesions 
have been directly imaged with "three-chip" digitizing cameras. Such cameras 
employ broad-band filter bandpasses that are ultimately based on the wavelength 
response of the human visual system and have large regions of overlap. 

25 Electronic images may also be obtained in narrower, non-overlapping 

filter bandpasses, which may reveal additional, wavelength-dependent differences 
between the images of melanomas and of benign lesions. However, such devices 
have had poor resolution and/or poor signal-to-noise characteristics which prevent 
the acquisition of digital images of melanocytic skin lesions of sufficient quality for 

30 effective application of machine vision techniques for lesion diagnosis. 

Existing imaging systems and processes also tend to suffer from an inability to 
provide the required repeatability of the values of extracted lesion parameters, due 
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in part to a lack of standardization with respect to spatially varying artifacts. The 
parameters, therefore, lack invariance to lighting and image exposure conditions, 
for example. Obtaining high signal-to-noise ratios in images recorded in narrow 
filter bandpasses, when exposure times are sufficiently short that the skin is 
5 effectively "frozen" during the exposure sequence, has also been difficult. In 

addition, since the optimum wavelengths for automatic characterization may not be 
the optimum wavelengths for visual observation, it may be difTicult to reconstruct 
high-fidelity color images from the digital Images for visual interpretation by a 
clinician. 

10 The assessment of wounds and bums through the appearance of 

color images present similar challenges. Existing technology for the imaging of 
skin in vivo for these purposes Is also inadequate. Practical solutions to the 
problems of employing multispectral digital imaging of skin for the analysis of 
lesions, wounds, or other conditions have not been found. 

15 SUMMARY OF THE INVENTION 

The methods and systems of the present invention provide for the 
acquisition of digital images of skin at a plurality of spectral bands to automatically 
characterize the condition of the tissue based on the digital images. Spectral 
wavelength bands within and outside of the visible band may be used. In 

20 accordance with the present invention, a pigmented skin lesion can be 

characterized as malignant or benign, for example. The digital images comprise a 
plurality of digital signals whose values are functions of the condition of the tissue. 
The digital images acquired are subjected to objective and quantitative analysis by 
a digital processor to detect and identify abnormalities. The analysis includes 

25 Image segmentation, parameter estimation and characterization of the skin. The 
estimation and characterization steps are automatic. The segmentation step may 
be automatic, as well. Subjective judgments are therefore minimized or 
eliminated. 

It has been found that generating the segmentation mask from a digital 
30 Image acquired with light In a spectral band which does not penetrate deeply Into 
the skin, such as a spectral band with a center less than about 500 nanometers. 
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provides superior results. After segmentation, estimated values which are 
functions of characteristics of the lesion, such as its texture, asymmetry, 
btotchiness, and border irregularities, are computed and used to automatically 
characterize the condition of the skin. Digital signals corresponding to hair or 
5 blob-like structures are preferably removed during segmentation. 

In accordance with the present invention, a method for characterizing 
the condition of a region of Interest of the skin, wherein the absorption and 
scattering of light in different spectral bands by the region of interest is a function 
of the condition of the skin, is disclosed. The method comprises illuminating the 

10 region of interest of the skin by light in at least three spectral bands and digitally 
Imaging the region of interest at the at least three spectral bands with the light re- 
emitted by the skin to generate digital images comprising digital signals whose 
values are a function of the condition of the skin. The digital Images are provided 
to a processor which segments the digital Images by generating a segmentation 

1 5 mask from a digital image in any one of the at least three spectral bands, 

computes at least one estimated value for each digital image at each spectral 
band which is a function of a characteristic of the region of interest within the 
segmentation mask, characterizes the condition of the skin based on the estimated 
values, and outputs the characterization of the condition of the skin. Preferably, 

20 the segmenting, estimating and characterizing steps are conducted without the 
intervention of an operator. Useful parameters include measures of the texture, 
asymmetry, blotchiness and border irregularity of the portion of the region of 
interest. 

The digital images may be obtained by directly imaging the region of 
25 interest with a digital camera, or digitally imaging color slides of the region of 
interest, through appropriately filtered light. 

The characterizing step may include comparing a weighted 
combination of the parameter values against a threshold value. The weight 
coefficients for each parameter value and the threshold value may be selected 
30 based on a training set of images of lesions or other skin conditions, whose 

condition has been determined, preferably through histological examination by a 
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plurality of doctors. Preferably, specificity is maximized under the constraint of 
1 00% sensitivity to melanoma. 

In accordance with another aspect of the invention, a system for 
characterizing the condition of a region of interest of skin Includes means for 

5 illuminating the region of interest with light in at least three spectral bands and a 
camera for acquiring digital images of the region of interest based on the light re- 
emitted from the illuminated region of interest at each of the spectral bands, the 
digital image comprises digital signals whose values are a function of the condition 
of the region of interest. A digital processor segments the digital images by 

10 generating a segmentation mask from a digital image in any one of the at least 
three spectral bands and computes at least one estimated value for each digital 
image at each spectral band which Is a function of the texture of the portion of the 
region of interest within the segmentation mask. The processor characterizes the 
lesion based on the estimated value or values. The other parameters discussed 

15 above may be used, as well. 

The camera may be a single-chip or multiple-chip charge-coupled 
device which detects light in a plurality of spectral bands between the near 
ultraviolet to near infrared. The filter means may be a plurality of interference 
filters mounted on a wheel for stepping any filter into a position intercepting the 

20 light from the light source. Preferably, at least one of the spectral bands has a 
center which lies between about 400 and 500 nanometers, at least one of the 
spectral bands has a center which lies between about 500 and 600 nanometers, 
and at least one other spectral band has a center which lies between about 750 
and 1000 nanometers. 

25 DESCRIPTION OF THE FIGURES 

Fig. 1(a) is a schematic illustration of a method and system of imaging 
a region of Interest of skin in accordance with the present invention; 

Fig. i(b) is a schenrtatic illustration of a plurality of narrow spectral 
bandwidths which may be used to illuminate the skin in the embodiment of Fig. 
30 1(a); 
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Fig. 1(c) is a schematic illustration of alternative methods and systems 
for digitizing and analyzing color photographic slides of a region of interest of skin; 

Fig. 2 is a schematic illustration of preferred illumination and imaging 
portions of a computer controlled Imaging system for direct imaging of a lesion; 
5 Fig. 3(a) is a flow chart of a calibration procedure for use with the 

present invention; 

Fig. 3(b) is a flow chart of a method of processing Images for 
classifying lesions as malignant or benign, in accordance with the present 
invention; 

10 Figs. 4(a) and 4(b) are histograms of a malignant melanoma and of an 

atypical melanocytic nevus, respectively, showing two peaks In each histogram; 
Figs. 5(a) and 5(b) are histograms of another malignant melanoma and another 
atypical melanocytic nevus, respectively, showing three or more peaks in each 
histogram; 

15 Figs. 6(a) and 6(d) are digital images in the blue spectral band of 

another malignant melanoma and another atypical melanocytic nevus, 
respectively; 

Figs. 6(b) and 6(e) are digital images of the images of Figs. 6(a) and 
6(d) respectively, after thresholding; 
20 Figs 6(c) and 6(f) are digital images of the images of Figs. 6(a) and 

6(d), respectively, after iterative thresholding; 

Figs. 7(a) and 7(d) are digital images in the blue spectral band of 
another malignant melanoma and another atypical melanocytic nevus; 

Figs. 7(b) and 7(e) are digital images of Figs. 7(a) and 7(d), 
25 respectively, resulting from iterative processing and showing dark blobs outside 
the lesion area; 

Figs. 7(c) and 7(f) are digital image masks of Figs. 7(b) and 7(d). 
respectively, resulting from image cleaning; 

Figs. B(a) and 8(e) are digital Images in the blue spectral band of 
30 another malignant melanoma and another atypical melanocytic nevus, 
respectively, showing hair; 
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Figs. 8(b) and 8(f) are reverse intensity contrast images of the lesions 
of Figs. 8(a) and 8(e), respectively; 

Figs. 8(c) and 8(g) are digital images resulting from an averaging 
process applied to the images of Figs. 8(a) and 8(b), to remove hair; 
5 Figs. 8(d) and 8(h) are binary lesion masks resulting from the 

segmentation of the images of Figs. 8(c) and 8(g)» respectively; 

Fig. 9 is a spatial filter used to remove hair; 

Figs. 10(a) - 10(c) are segmented digital images in the blue, green 
and red spectral bands, of the malignant melanoma whose histogram is shown in 
10 Fig. 5(a); 

Figs. 10(d) - 10(f) are segmented digital Images in the blue, green and 
red spectral bands, of an atypical melanocytic nevus whose histogram is shown in 
Fig. 5(b); 

Fig. 1 1 is a chart of lesion parameters and their associated diagnostic 
15 accuracy, sensitivity and specificity when used Individually; 

Fig. 12 is a plot of linear classifier values versus lesion identification 
number, for 41 malignant melanomas and 104 atypical melanocytic nevi; 

Fig. 1 3 is a plot of linear classifier values versus lesion identification 
number for 24 superficial spreading melanomas and 16 melanomas in-situ; and 
20 Fig. 1 4 is a plot of lesion parameter versus Breslow thickness for 24 

superficial spreading melanomas. 

DESCRIPTION OF THE INVENTION 
Fig. 1 (a) is a schematic Illustration of a method and system 1 in 
accordance with the present invention, by which images of the skin 2 are acquired 
25 by a camera nearly simultaneously at a plurality of spectral bands. Xh, 1=1 ,2,...M. 
that are preferably effectively non-overlapping, as shown schematically in FIG. 
1(b). the skin is illuminated by a source of white light 3, which is filtered by 
narrow passband filters 4. The filtered light is preferably conveyed to the skin 2 
through a fiberoptic illuminator 5. The light re-emitted by the illuminated skin is 
30 imaged by a low-noise, high-resolution monochrome camera 6, which Is preferably 
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an electronic charge-coupled ("CCD") camera. Digital images output by the 
camera 6 are provided to a computer 12 for processing. 

The computer 12 includes a digital interface 12a, a memory 12b and a 
digital processor 12c. A display 19 is preferably provided as well. The computer 

5 12 includes an input to a digital interfacie 12a for receiving the digital Images. A 
memory 12b stores the digital images, and the software controlling operation of 
the imaging system, the image processing, and the classification and 
characterization of the lesion. The digital processor 12c, under control of the 
software, performs the calculations. The computer 12 has an output connected to 

10 a display 19, which can display the processed images and the results of the 

classification and characterization procedures for each image. The computer 12 
also preferably has outputs connected to the source of light 3 and the camera 6, 
for controlling their illumination level and exposure times, respectively, as 
described below, 

15 The image processing, classification or characterization and other 

programs can be implemented on a personal computer, using a programming 
language, such as FORTRAN or C. The memory 12b which stores the software 
can be any convenient media readable by the computer 12, such as the hard drive 
of the computer, read only memory, random access memory with a battery backup, 

20 electrically programmed ROM, electrically erasable ROM, floppy disc, or CD ROM. 
Other suitable media may be used, as well. 

When the filter bandpasses have minimal overiap, as In FIG. 1(b), 
each monochromatic image will contain spectrally independent information. Such 
spectral separation is believed to be useful for differential diagnosis of skin lesions 

25 that contain varying amounts of melanin, and of hemoglobin in different oxidation 
states, for example. Spectral separation is also believed to be useful in 
distinguishing granulation of tissuid and other structural details of wounds in 
various stages of healing. One or more of the wavelength bands may lie outside 
the visible region, such as In the near infrared and/or the near ultraviolet, as long 

30 as the wavelength is within the response range of the combined optical system 
including the electronic camera 6. 
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In accordance with another aspect of the invention, the digital images 
of skin lesions can be derived from color slides of the lesions obtained by clinical 
imaging, dennoscopy, or polarization imaging. Fig: 1(c) is a schematic illustration 
of alternative approaches to the acquisition and digitization of images of skin 
5 lesions from color slides. A photo camera 13 produces 35-mm color slides of a 
region of the skin 14, The camera 13 can be a Dermaphot ® camera from Heine, 
Optotechnik Gmbh & Co. AG, Germany, for example. The slides are typically 
stored in an archive 15, The slides are subsequently reimaged by a monochrome 
camera 16, which may be a CCD camera, that photographs each slide as it Is 

10 illuminated by white light that has passed through a sequence of bandpass filters 
17 to create a color filtered version of the image. The slides can be illuminated at 
broad or narrow blue (B), green (G) and red (R) wavelength bands, respectively. 
The broad wavelength bands may overlap somewhat. In one example, the blue 
wavelength band was about 400 nm ± 30 nm, the green wavelength band was 

15 about 550 nm + 30 nm, and the red wavelength band was about 700 nm + 30 nm. 

Each of the filtered representations is recorded by the monochrome 
camera 16. which provides the resulting digital images 18 to an input of the 
computer 12. If an electronic camera such as a CCD camera is not used, the slide 
images could be digitized by any available commercial digitizer including three 

20 channels, one for red, one for green and one for blue, as long as the pixel size in 
the lesion plane after digitization is less than about 60 micrometers (Vm"). 

An appropriate CCD camera 16 is available from Electrim, Inc., 
Princeton, N J. The camera 16 has a photographic macro-lens, wherein t#/2.8 and 
f=100 mm. Preferably, the spatial resolution of the CCD camera 16 provides 

25 pixels having a size about 1 0-30 jim In the lesion plane. The CCD camera 16 from 
Electrim, Inc., has 753X488 pixels. The spatial resolution with such a camera is 
approximately 21 x 24 [xm at the lesion plane. Digital images of lesions obtained 
with this imaging systehi were used to classify lesions as malignant or benign, and 
to characterize lesions as invasive or non-invasive, as described further, below. 

30 The Electrim, Inc.. CCD camera 16 has rectangular pixels. A CCD camera with 
square pixels would simplify the calculating procedures! 
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Alternatively, a 3-chip CCD camera 20, indicated in phantom in Fig, 
1c, may be used to reimage the slides of the region of interest. The CCD camera 
20 provides digitized images for subsequent analysis by the computer 12. Broad 
bandpass filters, which are part of the CCD camera 20, produce a representation 
5 of the lesion as a set of three narrowband images. The filters are typically in 
accordance with CIE Standard Observer, wherein the bandwidths are broad. 

Fig. 2 is a schematic illustration of the illumination and Imaging 
portions of a preferred computer controlled imaging system 22 in accordance with 
the present invention, for imaging a region of interest of skin including a lesion. 

10 The electronic camera 23 may be a 10-bit monochromatic electronic CCD camera 
23, such as the Xilllx Model 1400, available from Xillix Technologies Corp., 
Canada. The Xillix camera is equipped with wide band, low distortion foreoptics, 
such as the XJP 1.9/0501, available from Jos. Schneider Werke, Genmany. The 
lower distortion fore optics and the camera minimize chromatic aberrations of the 

1 5 optical system over the "multlspectrar sequence of exposures, enabling 

registration of images with sub-pixel accuracy, over the entire field of view. 
To ensure repeatability of imagirig conditions and minimize required intervention 
by the operator, it is preferred that the system be operated at a preset f/stop. For 
cameras such as the Xillix Model 1400, exposure times are preferably controlled 

20 by the computer 12 through an electromechanical shutter that can operate reliably 
between minimum and maximum exposure times tmm and tnax- 

The imaging system provides low-noise, high-resolution digital images 
at high data transfer rates, with low distortion imaging over the entire range of 
wavelengths covered by the collection of filters. The Xillix camera, discussed 

25 above, has a resolution at the skin surface of about 20 microns per pixel. 

The CCD camera 23 is preferably contained in a hand-held unit, represented 
schematically as box 24. The illuminator source 25 is a tungsten-halogen lamp 
whose intensity is controlled by a light-stabilized power supply 26 whose setting is 
automatically adjusted by the computer 12. A 150 watt lamp, such as the Phillips 

30 EJA, available from Phillips Electronics North America Corporation, N.Y., may be 
used, for example. The output of the lamp 25 is white light A narrowband filter 27 
is provided between the source and an optical fiber 28. A plurality of narrowband 
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filters, each one corresponding to a different spectral wavelength band, are 
mounted on a filter wheel 29. Preferred filter bandwidths are listed in Table 1. 
below. The filter wheel 29, which is driven by a stepping motor 29a, advances 
each filter to its proper position between the lamp 25 and the optical fiber 28, and 

5 holds each filter in position for a sufficient period of time. The computer 1 2 

controls the motor 29a. l\^ore or fewer filters may be used. Appropriate lenses 14 
are provided between the lamp 25 and the filter 27, and between the filter 27 and 
the optical fibers 28, as well. One or more fiber Illuminators are provided for 
conveying the light from the source to the lesion. Two such illuminators 30a, 30b 

0 are shown in Fig. 2 for simplicity. Although the fiber illuminator illustrated is a 
bifurcated pair, a ring illuminator which provides more nearly uniform illumination 
at the skin surface, is preferred. An angle of illumination of about 20" is also 
preferred. A Fostec Model A0603 ring illuminator, available from Fostec, Inc., 
N.Y., may be used, for example. 

5 The hand-held portion of the system 24 of Fig. 2, which includes the 

camera 23, may be mounted on a cantilevered arm (not shown) that can be locked 
. into position. 

The digital signals making up each of the digital images output from 
the camera 23 are provided to the computer 12. The computer 12 conducts image 
0 processing procedures on the digital images to calibrate the images, and to 

objectively segment, estimate parameters, and classify the lesions based on the 
estimated parameters. Operator judgment is not required at any point in the 
process. 

Control is maintained by the computer 12 over source intensity, filter 
5 position, and such camera settings as shutter timing, through the digital Interface 
12a. Key control parameters ard empirically chosen on the basis of feedback from 
histograms of trial images. The intensity of the lamp 25 may be maintained at a 
stable value, commensurate vwth the 10-blt dynamic range of the camera 26, by 
monitoring a secondary light source, connected electrically in series with the 
0 primary light sdurce 25. The light output from the secondary source is monitored 
by a light sensor that is optically isolated from light reflections associated with the 
primary source. Such reflectionis may be caused by the filters that are located on 
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the filter wheel, or from the housing of the primary light source. This method 
provides optical feedback which is sensitive to changes in light interislty caused by 
changes in lamp lead resistance, for example, while It is insensitive to the variable 
amounts of light reflected from the filters, for example. By means of a closed 
5 control loop, the optical feedback from the secondary source is used to majntain 
long-term constant light output from the primary source. 

The apparatus of Fig. 2 ran be used for either clinical imaging of the 
skin, wherein the skin is imaged directly, dermoscopic imaging, wherein a layer of 
oil is provided over the skin and a layer of glass placed over the oil layer, or 

10 polarized Imaging, where a polarizer 31 is added to minimize specular reflection 
as shown in Fig. 2, In dermoscopic imaging, the index-matching oil sufficiently 
reduces the specular reflection to avoid the need for a polarizer. 
Instead of being positioned between the light source 25 and the optical fiber 28, 
the narrow bandpass filters 27 may be placed between the skin and the CCD 

15 camera 23 to filter the light reflected, scattered and radiated from the skin 2. 

The front end of the system preferably consists of a flat glass plate (not shown) for 
being placed over the skin. Light pressure is applied through the glass, onto the 
skin, throughout the imaging process. This helps to stabilize the region of interest 
against unwanted motion which could blur an image or which could lead to spatial 

20 misregistration between images obtained in different filter bandpasses. 

The prefen-ed filters 27 for lesion imaging with a tungsten-halogen 
white light source 25 have the center wavelengths and bandwidths (FWHM) 
listed in Table 1, for i=l,2„...M, M=IO, wherein the bands are labeled by j=i-l = 
0,1,,.. M-l. Such filters are available, for example, from Intor, Inc., Tucson, AZ. In 

25 each band, the exposure time is preferably selected to avoid saturation of the 
detector elements of the CCD camera 23. as well as to maximize the linear 
dynamic range over which thei image data are recorded. These exposure times 
should be constrained to be within limits tn,i„and tmax which are related to the 
electromechanical design of the shutter, optical throughput of the camera 23, and 

30 avoidance of Image blur associated with motion during the exposure sequence. 
Suitable values of tmm and tmax could be 10 ms and 550 ms, respectively, for 
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example. The choice of center wavelength and FWHM for the filter channels, as 
well as the corresponding exposure times, should preferably also take into account 
the following considerations: 

(a) The center wavelength and FWHM for at least two channels 

5 should be chosen so that characteristic absorption lines can be differentiated, 
such as those associated with melanin and hemoglobin; 

(b) For a given set of center wavelengths, there are upper limits on 
the associated bandwidths if spectral independence of data in different channels is 
to be maintained, as illustrated in FIG. 1(a); 

10 (c) Bandpasses should be chosen in the red, green and blue portions 

of the spectrum which enable "true-color** reconstruction of skin images that are 
suitable for visualization by clinicians; 

(d) The need for high signal-to-noise ratio in each image sets practical 
lower limits on the product of exposure time and filter bandwidth, especially at 

15 short wavelengths, where detector response fails off and lesion reflectance is low; 
and 

(e) The total time taken to acquire the images in all filter bands is 
preferably less than about three minutes, to minimize patient discomfort and 
possible motion. 

20 Based on considerations (d) and (e) above, and also taking into 

account the varying spectral reflectances of skin of different colors, the exposure 
times in each filter channel are preferably adjustable, with settings based on the 
dynamic range achieved on an empirical basis, with trial images. In this manner, 
both dynamic range and signal-to-noise ratio can be maximized for each filter 

25 channel. The preferred method is to choose t^xpi by iteration, based on intensity 

histograms of images of the skin obtained with trial exposures at each wavelength 
band. The histograms are analyzed to determine the number of pixels at the 
saturation intensity level, lsat= 2''-1 (1023 for b=IO bits). The exposure time is 
decreased if the number of saturated pixels exceeds a predetermined amount, 

30 such as 0.01% of the total. Conversely, to maintain high signal-to-noise ratio, the 
exposure time is increased if a predetemiined percentile in the histogram, 99.9%, 
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for example, is reached at less than a preset threshold, such as 99,5% of Isat. The 
iteration process typically converges after two or three trials. 

The preferred exposure times at each wavelength for imaging skin of 
different colors to classify melanomas are listed in Table 1. for the embodiment of 

5 Fig. 2 with 10 filters. It has been found that for the blue channel centered at 450 
nm, the optimal exposure time for dark skin is 273 ms, which is more than double 
the optimal 107 ms exposure time for light skin. On the other hand, in the near 
infrared channel centered at 780 nm, the exposure times listed are much shorter, 
between 24 and 35 nis, and vary relatively little with skin type. The optimal 

10 exposure time for dark skin in the deep blue channel at 430 nm is at tmax = 550 ms, 
due to the low skin reflectance and relatively low optical throughput of the system 
at this short wavelength. Even with an exposure time this long, therefore, the 
image Is less than fully exposed. Greater throughput at this wavelength could be 
achieved, at the expense of poorer response in the infrared. 

15 In Table 1 , the FWHM at 450 nm, is 1 00 nm, which is much broader 

than for other wavelengths. It has been found that where images are desired for 
visual analysis as well as computer processing, the broad wavelength band at 450 
nm more closely matches the blue response of the human eye and is therefore 
preferred. In addition, the broad wavelength band provides data at a higher 

20 signal-to-noise ratio. 

Table 1 appears below: 
Optimal Exposure Times (msec) vs. Skin Color 
Filter Center Filter 



25 


Number 
(i=i-l) 


Wavelength 
(nm) 


FWHM 
(nm) 


Very Light 
Skin 


i\/ledium 
Sl<in 


Tan 
Sl<in 

484.4 


Darl< 
Skin 




0 


430 


60 


405:2 


436.5 . 


550.0 




9 


450 


100 


106.8 


124.9 


156.1 


273:3 




1 


500 


40 


56.4 


62.9 


88.7 


130.7 


30 


2 


550 


10 


44.2 


50.4 


71.3 


92.9 




3 


600 


10 


19.1 


24.0 


29.6 


39.2 




8 


650 


10 


74.5 


92.3 


104.9 


132.0 




4 


700 


10 


71.6 


86.0 


98.0 


114.4 



SUBSTITUTE SHEET (RULE 26) 



Printed from Mimosa 



wo 98/37811 



PCTAJS98/03826 



16 



5 


780 


30 


25.8 


29.1 


34.9 


23.6 


6 


880 


50 


34.1 


38.6 


44.8 


46.1 


7 


950 


60 


161.1 


187.6 


205.8 


212.0 



Tables similar to Table 1 can be readily constructed based on 

5 experimental results for other applications, where other spectral bands may be 
better suited. For example, in the analysis of wound healing, the ability to 
distinguish oxygenated from deoxygenated blood would be desirable. In addition, 
the wavelengths and exposure times in Table 1 reflect a balance between the best 
results for subsequent analysis of the images by a computer, and the best results 

10 for visual observation of the images. If visual observation is not necessary, other 
wavelength bands and exposure times may be used. 

FIG. 3(a) describes how the systems and methods of the present 
invention provide for calibration of the recorded images. The calibration procedure 
permits 10-bit Image data to be reconded over a large linear dynamic range in 

15 each spectral band, independent of skin type. The recorded images can also be 
standardized for diffuse spectral reflectance. Consistent measures of reflectance 
ratios in different spectral bands can therefore be obtained, despite variations in 
illumination pattern with wavelength, changes in the position of the illuminator, or 
aging of the lamp, for example. 

20 First, the effects of dark current and "fixed pattern noise" are removed 

in Step 1 . N images are recorded by the camera without illumination. Preferably 8 
such dark images are recorded. The average of these N dark images, Id is 
calculated and stored in the computer 12. 

Second, spatial inhomogeneities In the illumination and In the 

25 response associated with each CCD pixel are removed in Step 2. A sequence of 
N' images of an illuminated flat, diffuse reflectance standard, such as a white 
Spectralon® target (R>99%) recorded. As above, N' is preferably 8. The N' 
images are recorded at each wavelength band. To average over local 
inhomogeneities in the reflectance standard, the target is moved continuously 

30 during the integration time and between exposures. A small motor, such as a 

reciprocating motor, may be used. The integration time and/or lamp intensity are 
adjusted by the computer 12 at each wavelength band until negligibly few of the 
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pixels are at or just below an intensity level corresponding to saturation. These N' 
"flat-field" images are averaged to reduce the effect of spatial non-uniformities in 
the reflectance standard, as well as to improve the detection signal-to-noise ratio. 
The resulting averages are stored in the computer as 1^1, where i=1,2,...M. 
Next, monochromatic "raw data" images of the skin, Isi, are captured by the camera 
and digitally acquired by the computer 12 within each filter passband. 1=1,2,. „M. If 
dermoscopic imaging is used, where a thin layer of mineral oil is spread between 
the skin and a cover glass is fixed In position in front of the camera, each image of 
the skin preferably contains an image of a narrow strip of oil-free, diffusely 
reflecting gray material, held in place on the inside surface of the cover glass, and 
located along one edge of the field of view. The material may be cut out of a 
Kodak "18% gray" card. Dermoscopic imaging is preferred for melanocytic 
lesions. The alternative clinical imaging mode is preferred for the imaging of 
wounds and bums because contact with the wound or burn by a cover glass is not 
desired. Although FIG. 2 indicates a lesion present on the skin 2, it will be readily 
understood that the same method will apply when a wound or burn is present, 
instead. In the clinical imaging mode, it is preferable to reduce specular 
reflections by employing the polarizer 31, as indicated in FIG. 2. 

In either the dermoscopic or clinical imaging techniques, a fourth step 
is preferably provided, in which the raw data is compensated for dark current and 
fixed pattern noise and then normalized to the flat-field images. The dark-field 
compensation is performed by subtracting the stored average dark image Id both 
from the flat-field image Ui and from the raw data image Ui . The ratio of the results 
of these subtractions is then taken. This standardizes the dark-corrected raw data 
to the flat-field Image, compensating for spatially varying illumination and pixel-to- 
pixel response variations. After the ratio is taken, the result is standardized to the 
maximum level, 2''-1 which equals 1023 where b = 10 in a 10-bit data 
representation. The normalization process thus converts the Image of the skin 
and the gray strip into a standardized diffuse reflectance map, with the result 
preserving a large linear recording dynamic range. In FIG. 3(a), the dark-field 
con-ected and flat-field-normalized images, also refen^d to as "fiat-field-caiibrated" 
images, are denoted as 1st. In any image, standardization to maximum level can be 
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reinterpreted directly in terms of equivalent diffuse reflectance on the basis of the 
average gray level over the image of the gray strip, <lgray strlp>i and the 
measured average diffuse reflectance of the gray strip, which is approximately 0.2 
and varies in a known and repeatable manner with wavelength. 
5 Preferably, the aveirage Image intensity In the gray-strip region is also 

used to calculate weighting factors for combining three or more monochromatic 
images to provide "true-color** visualizations of lesion images on the computer 12 
and display 19. This is preferably accomplished In Step 5. where the user selects 
the spectral bands to be used in the color visualization. Step 5 can take place 

10 prior to the imaging session. Four bands are currently preferred for such 

visualization. Filter bands j=3 and 8. in a 3:2 ratio, for the red (R) channel, filter 
band j=2 for the green (G) channel, and filter band j=9 for the blue (B) channel, in 
Table 1. As Indicated In Step 6 of FIG. 3(a), the relative weights applied to the 
R:G:B channels are preferably inversely proportional to <lgray strip>i the average 

15 intensity over the portion occupied by the gray strip area in each image. This 
procedure tends to reconstruct the hues and saturations in the original scene to 
within accuracy limite associated with response nonlinearities of the display 19. 
To minimize the effects of such nonlinearities with display monitors such as the 
Sony Model GDM-175E1 Multiscan monitor, for example, the viewer may prefer to 

20 adjust the maximum brightness in the image to correspond to the maximum image 
intensity level of the monitor. A linear transformation step, which can be readily 
accomplished by commercial software such as Adobe Photoshop, may be used. If 
the digital images are derived from photographic slides, as In the embodiment of 
Fig. 1(c), steps 5 and 6 are not necessary. 

25 As indicated by dashed lines in FIG. 3(a), either the normalized 

monochromatic images resulting from Step 4 or the color visualization provided 
from Step 6 can be displayed on the display 19. Any or all of the monochromatic 
raw images could be displayed as well. 

Fig. 3(b) is a flow chart of a preferred method of processing images 

30 according to the present invention for characterizing the condition of a region of 
interest of the skin of a subject which includes a skin lesion. A skin lesion is 
selected In Step 50. Digital Images of the lesions Illuminated by light filtered at the 
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desired wavelengths of Xi -X4..., are digitally recorded in Steps 52, 54, 56 and 
57..., as described above. Each of these digital Images is processed separately. 
In Step 58, the image taken at the lowest spectral band is used to create a mask 
for segmentation. A blue spectral band is preferred. At Steps 60, 62, 64, 65..;, 
5 each of the images of the lesion that correspond to different wavelengths are 
segmented by means of the segmentation mask obtained at Step 58. Lesion 
parameters are computed from each of the segmented images, in Step 66. Lesion 
parameters found to be useful for classifying and characterizing the lesion and 
statistical methods for computing the estimated values of the parameters, are 

10 discussed further, below. The estimated values of the parameters are provided to 
a linear classifier in Step 68. The linear classifier employs a linearly weighted sum 
of the individual parameters to derive a value used to classify the lesion as 
malignant or benign. A non-linear classifier such as a Gaussian quadratic 
classifier or an artificial neural-net classifier, each employing a suitable defined 

15 merit function, may be used as well. In either case, the numerical value produced 
by the classifier is subjected to a threshold test at Step 100, such that if the test is 
passed, the lesion is suspected to be malignant melanoma. If the test is failed^ the 
lesion is declared not to be melanoma. The lesion could also be characterized as 
invasive or non-invasive with a different classifier. 
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I. SEGMENTATION 

The segmentation algorithms wiU now be described. The function of the segmentation 
algorithms is to discriminate between the lesion and normal skin in the field-of-view of the 
imaging device. This b a complex function since not only is the lesion appearance highly 
variable but so is the appearance of healthy skin due. for example, to the presence of 
blotches, hair, wrinkles, etc. The automatic algorithm described here is based on the 
images in the blue spectral band, from about 400 nanometers (nm) to 500 nm. This 
spectral band was selected because melanin absorption increases rapidly as the wavelength 
decreases. While the use of ultraviolet radiation could be advantageous, since ultraviolet 
radiation is carcinogenic, only low doses can be used. 

Segmentation in blue consists of several automatic steps: 

Location of major pieaks in the histogram 

First, the histogram of intensity leveb in the whole image is determined. Then, given 
a sliding window with the range of (27^ + 1) intensity levels, the number of peaks V in 
the histogram over that range is determined. If AT, < 2, the range is decreased by two 
levels and if JV, > 3, the range is increased by two levels and the process is repeated untU 
ATp = 2 or 3. For most of the images in the data base used in this study, there are two 
major jpeaks in the histogram. Examples of such histograms are shown in Fig 4(a) for 
a maHgnant melanoma and in Fig. 4(b) for an atypical melanocytic nevus. The lesions 
correspond to the lower intensity peak, since it is darker than the surrounding skin due to 
strong absorption by melanin at 400 mn. However, some lesions are quite inhomogeneous 
and the automatic procedure described can find 3 major peaks, as illustrated in Figs. 5(a) 
and 5(b). 

Location of the intensity threshold 

If two major peaks are found in the intensity histogram, then the threshold value 
IS selected to be at the histogram minimum between these two peaks, as indicated in Figs 
4(a) and 4(b). In the case of three peaks, it has been found that, if the middle peak is closer 
to the lowest intensity peak, the threshold value is at the minimum b<?tween the middle 
and the highest intensity peak. If the middle peak is closer to the highest intensity peak 
then, the threshold value is at the minimum between the middle and the lowest intensity 
peak, as shown in Figs. 5(a) and 5(b). 

Iterative thresholding of the image 

The next step in image segmentation is iterative thresholding of the images. Given the 
intensity threshold value, image threshoiaing has been typically accomplished as follows. 
The mtensity /(*, y) of a pixel at location (x, y) is set to zero if it exceeds 7,^, i.e.. 
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Figs. 6(a) and 6(d) are examples of digital images of malignant melanoma and atypical 
melanocytic nevus in the blue spectral band, respectively. Figs. 6(b) and 6(e) are images 
resulting from the direct thresholding as in Eq. (1). As shown in Figs. 6(b) and 6(e) 
"holes" can appear within the lesion. Therefore, an iterative approach is preferably used' 
First, the intensity of pixeb at the image edges is set to zero. Then as each iteration 
proceeds, the intensity /(x,y) of a pixel at location (x,y) is set to zero if it exceeds I,^ 
and at least one of its nearest neighbors has zero intensity, i.e., 

^LK',y) otherwise, (2) 

where 

N„„ = nun(/(x - 1, y), /(x + 1, y), /(x, y - 1), /(I, y + 1), j . (g) 

This procedureis iterated untU there are no pixels with J(x.y) > and a nearest neighbor 
with zero intensity. Typically, only a few iterations are required to complete this step The 
resulting images are shown in Figs. 6(c) and 6(f). 

Figs. 7(a) and 7(d) are other examples of digital images of maUgnant melanoma and 
atypical melanocytic nevus, respectively. Figs. 7(b) and 7(e) are images resulting from the 
Iterative thresholding described above. Various dark blobs are seen in the images outside 
of the lesion area. These are removed in the following step. 

Image cleaning 

Some of the blobs in the thresholded images arise naturaUy due either to dark spots on 
the normal skm or to hair as in Fig. 7(b). Others are artifacts such as the film edge at the 
top of the nevus image in Fig. 7(e), or dark bands at the image edges from the sHde mounts. 
These bands are removed by automatically testing for their presence and then setting the 
intensity of appropriate pixels to zero. The remaining blobs could also be removed by 
determining the overall number and size, i.e;, number of pixels, of connected blobs, and 
then «5ttmg to zero the intensity of pixels belonging to the small ones. However, since the 
«ze of some lesions exceeds 100,000 pixels, this would be computationally very intensive 
Therefore, m practice, this step is preferably carried out as foUows. First, perimeter pixels 

1 n rmn ^i"^' ^^^^ '^^'^ """^^ P^*^ " typically less than 

10,000. Then, each of these perimeter pixels is assigned to a unique blob and its size the 

^umber of perimeter pixels in the blob, is determined. The intensities of pbcels belonging 
to blobs of size less than 30% of the maximum size for that image are set to zero This 
process is iterated until all the small blobs are removed. Typically less than 10 iterations 
are needed. The intensity of aU the nonzero pixels is then set to 1. The resulting binary 
lesion mask has the following property: 



jg^g^ y) / 1 . »f pixel at (x, y) belongs to lesion ; 
\0, otherwise. 



(4) 
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FigB. 7c) and 7(f) illustrate the resulting lesion masks. 

In the images iUustrated in Figs. 7(a) and 7(d), dark hairs were either absent or were 
not adjacent to the lesion. However, there are many images with prominent dark hair 
overlapping lesions. Segmentation of such images is described in the foUowing section. 

Segmentation of images in presence of hair 

Figs. 8(a) and 8(e) are examples of lesion images with hair. Since the segmentation 
algorithm described in the previous section would leave some of these dark hairs connected 
to the lesion, images with hair require special preprocessing to allow for hair removal from 
the normal skin. Since hair is a problem because of its high contrast with respect to the 
normal skin in the blue, a spatial filter was designed to locate hairs. This filter, shown 
in Pig. 9, is magnification dependent. It is appUed to every pixel of the original image 
and the result is thresholded at the 5% of maximum value in the whole filtered image 
The filtered images are shown in Figs. 8(b) and 8(f) in reverse intensity contrast, wherein 
bright features are dark. Hairs are clearly located in the filtered images. It should be noted 
that the lesion interior is ahnost entirely blank, indicating poor contrast between hair and 
lesion. 

Hairs are removed by an averaging process. For every non-zero pixel at (x, y) in the 
filtered image one finds the locations of 4 nearest pbcels (x,,y). (,,,y), (x,y,), 
(where xt < x < x« and yi < y < y,) with zero intensity. Then the intensity of every 
pixel in the original image that has non-zero intensity in the filtered image is replaced as 
follows: 

In(x,y) = 12 + y) + /(x/ 'k,y) + /(x,y. + k) + /(x,y, -k)] . (5) 



The images averaged in this way are shown in Figs. 8(c) and 8(g). It is seen that the 
contrast between hairs and normal skin is considerably reduced in these images. After this 
preprocessing, the segmentation algorithm described in the previous section is applied to 
the averaged image. The final binary lesion masks are shown in Figs. 8(d) and 8(h). 

The preprocessing step described above may be used for aU lesion images, regai-dless 
of the presence of hair, enabUng fully automated lesion segmentation. However, since this 
reqmres more computation and causes some border blurring, the need for preprocessing 
due to the presence of dark hair is preferably indicated interactively by an operator, and 
images preprocessed only when necessary. 

Segmentation of iniages in other spectral bands 

Since melanin absorption is strongest in the shortest-wavelength band, the lesion 
area, which appears as a dark region in the image, appears largest in the blue spectral 
band. Smce longer wavelength radiation penetrates deeper into skin, if the thickness 
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of the melanin-containing layer compensates for the weak absorption, that part of the 
lesion will appear dark even in the red spectral band. For thick melanomas, with Breslow 
thickness greater than 1 mm, one expects dark lesions even in the infrared bands. This 
was observed, for example, by Marchesini ci aL, Photochemistry & Photobiology, "In vivo 
spectrophotometric evaluation of neoplastic and non-neoplaatic skin pigmented lesions. III. 
CCD camera-based reflectance imaging," Vol. 62, 1995, pp. 151-154. However, for early 
malignant melanomas, with Breslow thickness less than 1 mm, great variability of images 
in the red spectral band has been foxmd. There may be so Uttle contrast between the lesion 
and the normal skin that direct segmentation is not possible. Therefore, segmentation of 
lesion images in aU spectral bands with wavelength A uses the binary lesion mask of Eq. 
(4), obtained in the shortest- wavelength band, here blue, i.e., 

^l{x, y; A) s I(x, y; A) x /^(z, y) . (6) 

Figs. 10(a) - 10(f) are a series of images of the lesions, with their corresponding 
histograms shown in Figs. 5(a) and 5(b), segmented in the blue, green, and red spectral 
bands, as indicated. The automaticaUy determined lesion borders are superimposed on 
the original lesion images. The area of dark regions is largest in the blue. 

n. LESION PARAMETER ESTIMATION 

Objective and automatic lesion classification requires quantitative algorithms for le- 
sion parameter estimation from their segmented images. Such parameters should be di- 
mensionless, independent of lesion location and orientation in the image, and of the overall 
image brightness. It is convenient to separate the parameters used here into four broad 
classes: asymmetry, blotchiness, border, and texture. Parameters with the highest diag- 
nostic accuracy for mahgnant melanoma are listed in Fig. 11, together with the values of . 
diagnostic accuracy, sensitivity, and specificity, for a training set of images of 41 malignant 
melanomas and 104 atypical melanocytic nevi obtained with the imaging system described 
above, with respect to Fig. 1(a) wherein the monochrome camera 16 was used to digitize 
sUdes. The subscript r, g, or b refers to the red, green, or blue spectral band in which the 
parameter is evaluated. If additional spectral bands are used, then each of the parameters 
could be computed at the additional spectral bands, as well. 

Specific algorithms for these parameters are described below. For simpUcity it is 
assumed that the image pixels are square but the algorithms described below may be 
implemented for rectangular pixels as well. ' 
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Lesion Asymmetry 
Asymmetry paramtier 

The lesion asynimetry parameter is based on moments of the intensity distribution 
First, the lesion orientation angle is used to locate the principal axes, which are just the 
symmetry axes for symmetric lesions. The angle « is computed from 

<(*-Xe)'>-<(y-ye)3>' (7) 
where the lesion intensity centroid is at 

Xe = < X > and ye = < y > . ^gj 

The angular brackets in Eqs. (7) and (8) denote an intensity moment, which for any 
function /(x,y) of position in the image can be computed as follows: 

</(x,y)>_ ^^2,/.(x.y) • W 

where Itix, y) is the segmented lesion image. In order to compare properties of different 
teions, the parameters used are independent of the orientation of the lesion in the image 
Therefore, the ledon asymmetry is determined with respect to the principal axes The 
measure of asymmetry described here requires rotation of the image by an angle 0 so that 
principal axes are parallel to the image axes. In this prindpal-axis coordinate system the 
loUowmg asymmetry factors are defined: 

t 

^ _ EnEy |i"r,(3:e+n,y)-/£,(xe-n,y)| 

A - E» En \^Ux, yc + n)- I dx,y^ - n)\ 
' : E.E,/L(x,y) • (lOi) 

The asymmetry pax ameter, 

A^ + Ay, (11) 
is a measure of asymmetry in the geometric shape of a ledon as weU as in the distribution 
of lesion pigmentation. Asymmetry parameters tend to be larger for maUgnant melanomas 
than for atypical melanocytic nevi. , 

Binary asymmetry parameter 

U the intensity distribution II in Eqs. (10a) and (10b) is replaced by the binary 
mtensity distribution of Eq. (4). then the corresponding asymmetry parameter ^».„ is 
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the fraction of the lesion pixels which do not have a coiinterpart on the other side of the 
principal axis. Thus, when based on the binary intensity distribution, parameter Ahin is a 
measure of the asymmetry of the geometric shape of the lesion. 

Lesion Blotchiness 

Visually, many early malignant melanomas appear blotchy. In multispectral images 
there may be darker and lighter regions or blotches of rather homogeneous intensity. In 
color images, in contrast, there may be regions of different colors. Therefore, it is of interest 
to quantify such blotchiness in order to differentiate malignant from benign lesions. 

Bloichinc3s Paramtitr Bastd on Spaiial InUiuiiy Disiribuiion 

The lesion is divided into Nt "topographic" regions. If /m«r and I^in are the maximum 
and minimum intensities in the lesion in some spectral band, respectively, then a pixel at 
(z, y) belongs to the nth region if 

For nth topographic region defined in Eq. (12), a distribution of distances of pixels in that 
region from the intensity centroid of the binary lesion mask 

^n(*. y) = \/(*n - Xc)' + iVn - Vc)^ (13) 

is obtained and its mean value < > and variance Var(dn) are computed. The measure 
of lesion blotchiness based on spatial intensity distribution is 

This parameter can be evaluated in every spectral band. 
Bloiekiness Parameter Based on Centroid^ 

The lesion is again divided into ATe "topographic" regions as defined in Eq. (12). An 
intensity centroid (xc(n), yc(n)), defined in Eqs/ (8) and (9), is then computed for each 
such region separately. The blotchiness parameter based on the centroid is defined as 

C = (Xmaz - Xmin){Ymaz - Ymin)/Al ' (15) 

where, for example, Xmax is the maximtmi value of ic(n), and Ai is the lesion area in 
pixels. This blotchiness parameter is also determined in each spectral band separately. 
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Bloichineis Parameier Based on Spatial Color Distribuiion 

The "color" in this analyaia is not related to the visual perception of color. It is 
a quantitative descriptor of the relative intensities in red, blue, and green channels in a 
particular pixel. 

AU the other lesion parameters described here involve analysis of images in each 
spectral band separately. Therefore, absolute calibration of image intensities was not 
necessary. However, in order to describe the color distribution, normalization of intensities 
in red, green, and blue spectral bands is needed, so that intensities in the three channels 
are equal for white. In the spherical color coordinate system, 

/2(x, y) = ^ ^ ^ -^^(''y) 

/R(«.y)+/B(x,y)+/c(x,y)' 

G(x v'\ = ^g(».y) 

' '^^ ^M*,y) + /B(x,y) + /c(x.y)' (16) 
where the subscripts R,G,B refer to red, green, and blue spectral bands, are chosen as 
the independent variables. The lesion is then divided into color regions as follows. First 
-R(x,y) and G(z,y) axe divided into Nr and Nc topographic regions. A color region is 
drfaed as a particular combination of two topographic regions. The total number of color 
re^qns is 



(17) 



The blotchiness parameter based on color is defined in analogy with Eq. (14): 

E-.<d„> • . W 

Lesion Border 

Border Irregularity Parameter 

Border irregularity is a well-known feature of maHgnant melanomas. It is typically 
defined as the ratio of the measured lesion perimeter to the perimeter of a circle with 
the saine area as the lesion. Since perimeter is difficult to estimate reliably, a statistical 
descriptor of border irregularity is used here. In addition, many lesions are elongated and 
an ellipse is a better approximation for such lesions with regular borders than a circle. 

Using the binary lesion mask of Eq. (4), the lesion intensity centroid from Eq (8) 
orientation angle from Eq. (7), area, and the aspect ratio defined as 



V< y'-yc>»' W 
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Whew pnma refer to the coordinate .yatem defined by the lesion principal axes ate 
detcnmned. These values are then used to construct an ellipse that is the best regular 
approximation to the lesion border. For each lesion border pixel at y*). its angle^th 
respect to the horizontal aods: 



^ = tan-»^fi:iis). 
(yt - »e) 



(20) 



and the location of the ellipse border for the same angle are determined 

The distnbutjon of distances between the eUipse border and lesion border: 



where 
and 

is obtained and the border irregularity parameter is defined as 



(22) 
(23) 



„ _ vryar(d.O 

.<d»> • (24) 



Border Gradient Parameter 

Another parameter that quantitatively characterizes lesion border is the measure of 
intensxty ^adients across the lesion borders over the length scale defined by n„ in units 
of p«ds. For each lesion border pixel at (x^.ys) one determines whether pixdl at (x» ± 
n„ y» ± n, ) are at the border. If they are not, then the gradient is defined 



; as 



G(z»,yO = il|/(x+n,,y)-/(x-n,.y)j + |J(x,y + „,)_/(,,y.„^^^^ (25«) 
otherwise, if pixels at (* ± n„ y) are not on the border. 

G(x»,yt) = |/(x + n„y) -/(*_„„ y)|, 
or, if pijcels at (x, y ± n,) are not on the border, 

G(ifc,y4) = |/(x,y + n,)-/(x,y->,)|. 
The border gradient parameter is defiiied as 



(256) 



(25c) 



<G> • (26) 
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Leaion Texture 

The description of lesion texture is particularly vulnerable to subjective judgement. 
The quantitative evaluation of lesion texture parameters is possible only using computer- 
based image analysis. While many such parameters are possible, those found to be helpful 
in discriminating between malignant melanomas and atypical melanoqrtic nevi are de- 
scribed below. 

Texture Parameters Based on Local Intensity Variations 

Texture parameters are defined over a length scale n, in units of pixels. For example, 
consider a pixel located at (*,y) in the lesion. Ut /f and U be the minimum and the 
maximum intensities in an image in the 2n, -I- 1 x 2n, -I- 1 window around this pixel, i.e., 
in the range (* - n,,x -|- n,) and (y - n,,y + m]. Considw a variable 

C,(x,y)^iiLZii. 
The first two texture parameters are defined as: 

<Ci>. ; (28) 

■and 

r2= VVar(C,) (39) 
Another texture parameter uses the following variable: 

<^3(x, y) = 4u)(0, 0) + w{-nu 0) 4- u>(n,, 0) -I- ti>(0, -nO -1- u>(0, n,) 
- 2(ti»(-ni, -n,) + u>(-n„ n,) + ti,(n,, -n,) -H u>(n,, n,)J ' 
where 

= /(* + »,y + j)//(*,y). (31) 
If the value of C3 is negative, it is set to zero and the corresponding texture parameter is 

<C3> • (32) 

Another variable that leads to a texture paranieter useful for dassiacation of melanocytic 
lesions is: 

C^4(*, y) = 8to(0,0) - ti>(-n„0) - u;(n„ 0) - «»(b, -n,) - w(0,n,) 

-ti>(-n«,-ne)-u,(-n«,nO-t»(n,,-n,)-u»(n,,nt) ^^^^ 
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Again, if the value of the variable is negative it is Mt to zero and the correapondiae texture 
parameter is ^ 

<C4> • (34) 
Texiure Pammtiera Biued on Pigmented Network 

Texture parameters have also been developed by considering the properties of a pig- 
mented network. These texture parameters are measures of variability in the area of the 
dermal papillae and in the aspect ratio (length/width) of the rete ridges. 

Since dermal papillae appear as the brighter part of the network, one seeks aU the 

local maxima over a 2n, + lx2n, + l window. Startingfrom such amaximum at (z„ y ) 
one finds local one-dimensional minima in eight directions (2 vertical. 2 horizontal' 2id 
4 diagonal) and locates the vertices of an octagonal region one pixel closer to the max- 
imum mtensity pixel than the minimum pixel. Such octagonal regions approximate the 
areas of dermal papillae Ai„ which are computed from the known location of vertices- the 
corresponding texture parameter is ' 

. <A,,> (35) 

Some of the areas determined by this algorithm are due to bubbles visible in some of these 
dermoscopic unages. However, since there are typically on the order of hundreds of areas 
and on the order of tens of bubbles, the statistical parameters should not be significantly 
biased by this artifact. 6"^««wy 

The aspect ratio of rete ridges is detennined in a similar fashion, although one starts 
with local minima since rete ridges appear dark in the images. The vertices of an octagonal 
region axe detennined in this case from one-dimensional maxima in the eight directions 
The maximum and minimum extents of this region are then determined and the aspect 
ratio J2 is computed. This texture parameter then is 



<R> ' : (36) 
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in. LESION CLASSIFICATION 

Sdection of lesion parameters for classification was done by detennining the maximum 
diagnoatic accuracy for malignant melanoma for each parameter computed in every spectral 
band available for the training set of images. As mentioned above, diagnostic accuracy, 
sensitivity to malignant melanoma and specificity for the selected twenty two parameters 
are shown in Fig. 11. These parameters were then used as input to the linear classifier. 
Nonlinear classifiers may be used as well. 

For each lesion k the linear classifier is 



3} 



(37) 



where Jt„(*) are the parameters for the kth lesion and weights u»„ are to be determined so 
that a specified function attains maximum value. The following functions F(L) were 
used: 1) specificity under constraint of 100% sensitivity to malignant melanoma for the 
training set which included 41 malignant melanomas and 104 atypical melanocytic nevi; 
(2) classification accuracy for the 24 invasive and 16 noninvasive maUgnant melanomas of 
the training set; and 3) correlation with the Breslow thickness for the 24 invasive malignant 
melanomas. 

Given any training set of lesion images and corresponding set of lesion image param- 
eters, the weights that maximize F(L) are found as foUows. First, an initial range and 
resolution A for ui„ are selected. For each allowed set of values of u;„, the values L{k) 
are computed for each lesion. The value FiL) is determined based on the input from 
histopathological evaluation of the lesion based on a biopsy, such as the diagnosis of the 
lesion as benign or malignant, and the Breslow thickness for a mahgnant melanoma. The 
range of ui„'s is adjusted untU the maximum value of F{L) is inside the range. Then the 
resolution A is reduced by a factor of two, and the process is repeated until A reaches 
spedfied minimum value A«i„. This procedure determines the weights u;„ only up to a 
multipUcative constant. It is noted that the classifiers resulting from a particular training 
set are applicable only to images with a specific spatial and spectral resolution, and that 
leion images obtained with a different imaging system may reqtrire the development of 
different classifiers, by the procedures d«cribed above- 
Since detection of melanoma in its early stage significantly improves prognosis, there 
IS a need for reUable methods of early detection. Clinical evaluation of melanocytic lesions 
IS, however, a problem since reliable differentiation between early malignant melanoma 
with Breslow thickness less than 1 mm and atypical melanocytic nevus is difficult even 
for experienced dermatologists. In order to detect as many early melanomas as possible, 
weights in the linear classifier are preferably chosen to maximize specificity under the 
constraint of 100% sensitivity to malignant melanoma for the training set. For each set 
of weights, one finds the threshold value L.* of the Unear classifier such that a ifeth lesion 
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is classified as suspicious of mulignaacy if > Xa. and benign otherwiac The 

resulting classifier for the training set is "inerwiae. The 

Li = 0.025 Aii„ + 0.090 At + 0.069 A, + 0.160 Ar + 0.128 C» 

+ 0.095C/ + 0.0385 + 0.107ri,+0.064r2, + 0.018r2, (33) 
+ 0.111 r34 + 0.167 r3, + 0.268 rst 

wh^e the weights are normaUzed so that the threshold value equals one. This classifier 
^th seusi mty to malignant melanoma of 100% and specificity of 85% is shown in Fi. 
12. Statistical «gi"ficance of the specificity and sensitivity was assessed by considering 
the bmomial probabahties for the value of X, to exceed the threshold for the 41 mah,^! 
mdanomas and 104 atj^ical melanocytic nevi of the training set separately. A't"^^ 

t W 79T S • r T'*'"*^ " ^^"^ ^"^'^ » not less 

than 79%. Since there are several melanomas very close to the threshold value; a practical 

cl^afier may use a threshold value that is less than one. It has been found that tL se Tf 

145 .magesjs suffiaent to yield statisticaUy significant results. A greater number of 

may be used» as well. ""»6cs 

J^oed to p^ocm this t«k. Thi, cla«,ifi„, with w^jht, chos« to m«i^"tt^ 

.^-trr :^:^rr -^-^^ « 

1^ = -1.00 Ai - 0.14 B:, - 2.47 B/r - 0.4 C» - 0.98 CZ 

- i.i7r2r + 0.53 r4» + 1.98X54 + 1.58 re* -0.73 

where a constant was subtracted from the classifier values to obtain the threshold value of 
zero. Thas classifier, with overaU classification accuracy of 92.5%. is shown in Fig 13 
a linl'rr P~^°l^f invasive melanomas correlates strongly with the Breslow thickness 
a^near fmction of lesion parameters Q was trained to maximize the Pearson correlat^Z 

t^rS^ ^ ^"^'"^ '^^-^ * --'^^^ ~« -lano^r, 

9 -0-955 A, - 1.391 Ar + 2.791 SZ» - 1.320 Pi, + 0.146 C» 
+ 0.267 Cr - 0.506 5 + 0.202 ri, + 1.476 r2r- 0.485 

^d « shown in Fig. 14. Even though there are only 24 superficial spreading melanomas in 
the data base, the high correlation of 0.77 is statisticaUy very significant since p = 9 x 10-. 
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The clas3ifiers of Eqs. (38)-{40) arc applicable to the imaging system described above, 
with respect to Fig. 1(a) wherein the monochrome camera 16 was used to digitize slides. 

For other imaging systems, having different spatial and spectral resolution, different 
classifiers may need to be developed, based on a sufficient data base of lesion images 
obtained with that imaging system, in accordance with the prpcedureis described above. 
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The segmentation, parameter estimation and classification programs 
described in Sections l-lll, above, can be implemented on any personal computer, 
using a programming language, such as FORTRAN or C. The program can be 
stored on any convenient media readable by a computer, such as read only 
5 memory, ("ROM"), random access memory with a battery backup, electrically 

programmed ROM (EPROM), electrically erasable ROM (EEPROM), floppy disc, 
CD ROM, or a hard disc. Other suitable media may be used, as well. 

While the procedures of Sections l-lll were described with respect to 
digital images obtained by imaging color photographic slides of skin lesions with a 

10 monochrome CCD camera 16 In accordance with the system of Fig. 1c, these 

procedures are readily adaptable to the analysis of digital images of skin lesions 
acquired directly from the region of Interest of the skin with a monochrome CCD 
camera 6 of Fig. la and Fig. 2. 

In the process described in Sections l-lll, above, segmentation was 

15 conducted in the blue wavelength band. The segmented mask in blue was then 
applied to images in the red and green wavelength bands. Where images at 
additional wavelengths are provided, segmentation is preferably conducted of the 
image at the shortest available spectral band where the contrast between the 
lesion and normal skin tends to be highest because of strong absorption by 

20 melanin. The segmented mask is then applied to the images In the remaining 
spectral bands. These steps are shown in Fig. 3b. 

In addition, where parameters are described in terms of red, green and 
blue wavelength bands in Section l-lll, parameters can be derived at each of the 
other wavelengths used in the systems and methods above, in accordance with the 

25 procedures described in Section l-lll. The additional parameters can be readily 
used to develop a classifier, also by the processes described in Section l-lll. 

In addition, where parameters are described In terms of red, green and 
blue wavelength bands In Section l-lll. parameters can be derived at each of the 
wavelengths used in ttie systems and methods above, In accordance with the 

30 procedures described in Section l-lll. The additional parameters can be readily 
used to develop a classifier, also by the processes described in Section l-lll. 
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The references cited above are incorporated by reference herein. 

While preferred systems and methods for practicing the present 
Invention have been described above, it Is understood that departures may be 
made from the systems and methods, without departing from the scope of the 
5 present invention, which is defined by the following claims. 
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We claim: 

1 . A method of characterizing a skin lesion wherein the 
absorption and scattering of light in different spectral bands by the skin lesion 
is a function of the condition of the skin, the method comprising: 

illuminating ai portion of the skin including the region of interest 
by light in at least three spectral bands, one of which is a blue spectral band; 

digitally imaging a portion of the skin including the region of 
interest at the at least three spectral bands with the light re-emitted by the 
portion of the skin to generate digital images comprising digital signals whose 
values are a function of the condition of the region of interest of the skin; and 

providing the digital images to a processor, wherein the 

processor: 

segments the digital image by generating a segmentation mask 
defining the boundary of the region of interest from a digital Image In the blue 
spectral band, without operator intervention; 

automaticaliy computes at least one estimated value for each 
digital image at each spectral band which Is a function of a characteristic of 
the portion of the region of interest determined by the segmentation mask, 
without operator Intervention; 

characterizes the condition of the skin as malignant or benign 
based on the estimated values, without operator intervention; and 

outputs the characterization of the condition of the skin. 

2. The method of claim 1 , further comprising estimating at least 
one value which is a function of the texture of the region of interest. 

3. The method of claim 2, wherein the computing step 
comprises estimating values which are statistical measures of local intensity 
variation in the digital images in each spectral band, which are a function of 
the texture of the region of interest. 

4. The method of claim 2, wherein the computing step 
comprises estimating values based on the ratio of standard deviation of the 
areas of dermal papillae to their mean within the segmentation mask. 
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5. The method of claim 2, wherein the computing step 
comprises estimating values of the average and standard deviation of the 
thickness of rate ridges within the segmentation masics. 

6. The method of claim 1 , further comprising estimating at least 
one value which is a function of the asymmetry of the region of interest in 
each spectral band, for two principal axes of the segmented image by: 

locating the principal axes by computing an orientation angle; 

computing the intensity centroid; 

rotating the digital image such that the principal axes are 
parallel to the image axes; and 

estimating the asymmetry values for each principal axis based 
on the intensity centroid; and 

summing the estimated asymmetry value for the two principal 

axes. 

7. The method of claim 1 , further comprising estimating at least 
one value which is a function of the blotchiness of the region of Interest. 

8. The method of claim 1, further comprising estimating at least 
one value which is a function of the irregularity of the border of the region 

of interest by estimating a value which is a statistical measure of the 
deviation of the border of the segmentation mask from the border of an 
ellipse of the same area, aspect ratio, and orientation as the segmentation 
mask. 

9. The method of claim 1 , further comprising estimating a value 
which Is a function of the gradient at the border of the region of interest by 
estimating a statistical measure of the gradient values of the intensity of the 
digital images across the border of the segmented images, at each spectral 
band. 

10. The method of claim 1, further comprising characterizing 
the type of lesion as invasive or non-invasive. 

1 1 . The niethod of claim 1 . wherein the segmenting step 
comprises generating the segmentation mask from a digital image by: 
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removing digital signals from tlie digital image which correspond 
to hair structures; 

deriving a threshold from a multimodal histogram of intensity 

levels; 

iteratively applying the threshold to the digital signals of the 
digital Image; and 

removing digital signals which correspond to small blob-like 
regions from the masked image. 

12. The method of claim 1 , wherein the digital imaging step 
further comprises digitally Imaging the region of Interest with a digital camera. 

13. The method of claim 1 , further comprising: 
photographing the region of interest with a color camera to form 

color photographic slides; and 

illuminating the color photographic slides with light in each 
spectral band; 

wherein the digital imaging step comprises digitally imaging the 
illuminated color photographic slides of the region of Interest with a digital 
camera. 

14. A method of characterizing the condition of a region of 
interest of skin, wherein the absorption and scattering of light in different 
spectral bands by the region of Interest is a function of the condition of the 
skin, the method comprising: 

illuminating a portion of the skin including the region of interest 
by light in at least three spectral bands; 

digitally imaging the portion of the skin including the region of 
interest at the at least three spectral bands with the light re-emitted by the 
portion of the skin to generate digital images comprising digital signals whose 
values are a function of the condition of the region of interest of the skin; and 

providing the digital images to a processor, wherein the 

processor: 



Printed from Mimosa 



wo 98/3781 1 PCT/US98/03826 

38 

segments the digital images by generating a segmentation 
mask defining the boundary of the region of interest from a digital Image in 
any one of the at least three spectral bands; 

computes at least one estimated value for each digital image at 
each spectral band which is a function of a characteristic of the region of 
interest determined by the segmentation mask; 

characterizes the condition of the region of interest of the skin 
based on the estimated values; and 

outputs the characterization of the condition of the region of 
interest of the skin. 

15. The method of claim 14, wherein the estimating and 
characterizing steps are conducted without the intervention of an operator. 

16. The method of claim 14, wherein the segmenting step is 
conducted without the intervention of an operator. 

17. The method of claim 14, wherein the illuminating step 
further comprises illuminating the region of interest with light in at least one 
spectral band which penetrates to the papillary deirmis and is re-emitted 
therefrom. 

18. The method of claim 17, wfierein the digital imaging step 
further comprises digitally imaging the region of interest with a digital camera. 

19. The method of claim 17, wherein the illuminating step 
further comprises illuminating the region of interest with light in a near 
infrared spectral band. 

20. The method of claim 14, further comprising suppressing 
specular reflections prior to the digital imaging step. 

21. The method of claim 20, wherein the processor converts 
the digital signals of each of the digital images into values corrected for non- 
uniformities of illumination and of response prior to the segmenting step. 

22. The method of claim 14, further comprising: 
photographing the region of interest with a color camera to form 

color photographic slides; and 
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illuminating the color photographic slides with light in each 
spectral band; 

wherein the digital imaging step comprises digitally imaging the 
illuminated color photographic slides of the region of interest with a digital 
camera. 

. 23. The method of claim 14, wherein the segmenting step 
further comprises applying the segmentation mask to the digital images in the 
other spectral bands. 

24. The method of claim 14, wherein the segmenting step 
comprises generating the mask at the shortest available wavelength. 

25. The method of claim 14, wherein the illuminating step 
comprises illuminating the region of interest by light in at least one spectral 
band whose center is between about 400 to about 500 nanometers, and the 
segmenting step comprises generating the mask from a digital image at the 
spectral band between about 400 to about 500 nanometers. 

26. The method of claim 16, wherein the segmenting step 
comprises generating the segmentation miask from a digital image by: 

removing digital signals from the digital image which correspond 
to hair structures; 

deriving a threshold from a multimodal histogram of intensity 

levels; 

iterdtively applying the threshold to the digital signals of the 
digital image; and 

removing digital signals which correspond to small blob-like 
regions from the masked image. 

27. The method of claim 16, wherein the segmenting step 
comprises generating the segmentation mask by coinparing estimated values ; 
which are a function of textures within the digital images with a threshold. 

28. The method of claim 27, further comprising generating the 
segmentation mask by comparing the estimated texture values to a threshold 
derived through statistical analysis of each digital image. 
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29. The method of claim 14, wherein the computing step 
comprises estimating at least one value which is a function of the texture of 
the region of interest determined by the segmentation mask. 

30. The method of claim 29, wherein the computing step further 
comprises estimating values which are a function of the texture of the region 
of interest determined by the segmentation mask separately in each spectral 
band, based on the same segmentation maslc. 

31 . The method of claim 29, wherein the computing step 
comprises estimating values which are statistical measures of local intensity 
variation in the digital images In each spectral band which are a function of 
texture. 

32. The method of claim 14, wherein the computing step further 
comprises estimating a value which is a function of the asymmetry of the 
segmented innage in each spectral band, for two principal axes of the 
segmented image. 

33. The method of claim 32, wherein the computing step further 

comprises: 

locating the principal axes by computing an orientation angle; 

computing the intensity centrpid; 

rotating the digital image such that the principal axes are 
parallel to the image axes; 

estimating asymmetry values for each principal axis based on 
the intensity centroid; and 

summing the estimated asymmetry values for the two principal 

axes. 

34. The method of claim 33, wherein the computing step further 
comprises computing the intensity moment with a binary intensity distribution. 

35. The method of claim 14, wherein the computing step further 
comprises estimating at least one value which is a function of the blotchiness 
of the segmented digital image, the estimated blotchiness value being 
defined through statistical properties of the spatial distribution of topographic 
regions of the digital irriages at each spectral band. 
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36. The method of claim 35, wherein the computing step further 
comprises determining the centroids of topographic regions of the segmented 
digital image at each spectral band. 

37. The method of claim 14. wherein the computing step 
comprises estimating a value which is a statistical hneasure of the deviation of 
the border of the region of interest from the border of an ellipse of the same 
area, aspect ratio, and orientation as the segmentation mask. 

38. The method of claim 14, wherein the computing step 
comprises estimating a statistical measure of the gradient values of the 
intensity of the digital images across the border of the segmented images, at 
each spectral band. 

39. The method of claim 14, wherein the computing step 
comprises estimating values based on the ratio of standard deviation of the 
areas of dermal papillae to their mean within the segmentation maslc. 

40. The method of claim 14, wherein the computing step 
comprises estimating values of the average and standard deviation of the 
thickness of rete ridges within the segmentation mask, 

41 . The method of claim 14, wherein the characterizing step 
comprises comparing a weighted combination of parameter values against a 
threshold value. 

42. The method of claim 41 , wherein the condition of the region 
of interest to be characterized is the presence of a melanoma and weight 
coefficients for each parameter value and the threshold value are selected to 
maximize specificity, under the constraint of 100% sensitivity to melanoma, 
on a representative set of training images 

43. The method of claim 14, further comprising calibrating each 
pixel location in the digital image in each spectral band with respect to stored 
images of a white target material having known diffuse reflectance, each of 
the stored images being an average of a plurality of images acquired at each 
spectral band, while the material undergoes continual in-plane motion. 

44. A system for characterizing the condition of a region of 
interest of skin, comprising: 



Printed from Mimosa 

i 



wo 98/37811 



42 



PCT/US98/03826 



a source of Illumination of light in at least three spectral bands; 

a camera for acquiring digital Images of the region of interest 
based on the light re-emitled from the illuminated region of Interest at each of 
the spectral bands, the digital image comprising digital signals whose values 
are a function of the condition of the region of interest; 

memory for storing the digital images provided by the camera; 

a digital processor programmed to perform the steps of: 

segmenting the digital Images istored in memory by generating a 
segmentation mask from a digital image in any one of the at least three 
spectral bands; 

estimating at least one value for each digital image at each 
spectral band which is a function of the texture of the portion of the region of 
interest determined by the segmentation mask; 

characterizing the condition of the skin based on the estimated 

values; and 

outputting the characterization of the region of interest. 

45. The system of claim 44, further comprising means for 
suppreissing specular reflections from the region of interest 

46. The system of claim 44, further comprising means for 
calibrating each digital image to provide correction for non-uniformities of 
illumination and response. 

47. The system of claim 44, wherein the digital processor is 
coupled to the source of illumination and to the camera for controlling the 
intensity of illumination and exposure times, respectively. 

48. The system of claim 44, wherein the processor applies the 
segmentation mask derived from the digital images at one spectral band to 
the digital images at the other spectral bands 

49. The system of the claim 44, wherein the processor 
estimates values separately from digital images at each spectral band based 
on the segmentation mask. 

50. The system of claim 48, wherein the processor compares a 
weighted combination of estimated values against a threshold value. 



Printed from Mimosa 



PCT/US98/03826 

43 

51 . The system of claim 44, wherein the camera records 
monochromatic images and the illumination means comprises: 

a tungsten halogen light source with feedback to stabilize the 
Intensity in each wavelength band; 

means for sequentially filtering the light; and 

an optical fiber ring illuminator to distribute the filtered light. 

52. The system of claim 44, further comprising a feedback loop 
for stabilizing the intensity of the light source by the processor 

53. The system of claim 44, wherein the filter means comprises 
a plurality of interference filters mounted on a wheel for stepping any filter 
into a position intercepting the light from the light source. 

54. The system of claim 44, wherein at least one of the spectral 
bands has a center which is between about 400 to about 500 nanometers, 
and at least one other band centered elsewhere in the visible region. 

55. The system of claim 54, wherein the set of interference 
filters includes a filter whose center lies in at least one spectral band in the 
near Infrared range whose center lies between about 750 and 1 000 
nanometers. 

56. The system of claim 49, wherein the camera is a single- 
chip, charge-coupled device and the control means comprises a digital 
computer including means for determining exposure times for the camera 
which maximize the signal-to-noise ratio in the image at each spectral band. 

57. The system of claim 44. wherein: 

the source of illumination provides broad-band ("white") light; 

and 

the camera comprises multiple charge-coupled devices which 
detect light in a plurality of spiBctral bands between the near ultraviolet to 
near infrared. 

58. The system of claim 44, wherein the processor estimates 
values which are statistical measures of local intensity variation in the digital 
images in each spectral band, which are a function of the texture of the 
region of interest. 
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59. The system of claim 44. wherein the processor estimates 
values based on the ratio of standard deviation of the areas of dermal 
papillae to their mean within the segmentation mask; 

60. The system of claim 44, wherein the processor estimates 
values of the average and standard deviation of the thickness of rete ridges 
within the segmentation masks. 

61 . The system of claim 44, wherein the processor estimates at 
least one value which is a function of the asymmetry of the region of interest 
in each spectral band, for two principal axes of the segmented image by: 

locating the principal axes by computing an orientation angle; 

computing the intensity centrold; . 

rotating the digital image such that the principal axes are 
parallel to the image axes; and 

estimating asymmetry values for each principal axis based on 
the intensity centrold; and 

summing the estimated asymmetry values for the two principal 

• axes: . 

62. The system of claim 44, wherein the processor further 
estimates at least one value which is a function of the biotchiness of the 
region of interest. 

63. The system of claim 44, wherein the processor further 
estimates at least one value which is a function of the irregularity of the 
border of the region of interest by estimating a value which is a statistical 
measure of the deviation of the border of the segmentation mask from the 
border of an ellipse of the same area, aspect ratio, and orientation as the 
segmentation mask. 

64. The system of claim 44, wherein the processor further 
estimates a value which is a function of the gradient at the border of the 
region of interest by estimating a statistical measure of the gradient values of 
the intensity of the digital images across the border of the segmented images, 
at each spectral band. 
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65. The system of claim 44, wherein the processor 
characterizes the type of lesion as invasive or non-Invasive. 

66. The system of claim 44, wherein the processor generates 
the segmentation masic from a digital image by: 

removing digital signals from the digital image which correspond 
to hair structures; 

deriving a threshold from a multimodal histogram of intensity 

levels; 

iterativeiy applying the threshold to the digital signals of the 
digital image; and 

removing digital signals which correspond to small blob-lilce 
regions from the masked image. 

67. A system for characterizing the condition of a region of 
interest of skin, comprising: 

a source of illumination of light in at least three spectral bands; 

a camera for acquiring digital images of the region of Interest 
based on the light re^emitted from the illuminated region of Interest at each of 
the spectral bands, the digital image comprising digital signals whose values 
are a function of tlie condition of the region of interest; 

a memory for storing the digital Images; 

a digital processor including: 

digital processing means for segmenting the digital images 
stored in memory and computing estimated values of parameters which are a 
function of the segmented images; 

digital processing means for automatically characterizing the 
condition of the tissue based on the estimated values; and 

means for outputting the characterization of the region of 

interest. 
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